Reconstruction of a high-resolution image from a sequence of lower resolution images is a way to increase the effective spatial resolution of a camera capturing conventional movie images (see, e.g.: Huang, T. S. and Tsai, R. Y., “Multi-frame image restoration and registration,” Advances in Computer and Image Processing, Vol. 1, (1984); Kim, S. P., Bose, N. K., and Valenzuela, H. M., “Recursive Reconstruction of High Resolution Image From Noisy Undersampled Multiframes,” IEEE Trans Acoustics, Speech, and Signal Processing 38 (6), 1990; Tom, B. C., Katsaggelos, A. K., “Reconstruction of a high resolution image by simultaneous registration, restoration, and interpolation of low-resolution images,” Image Processing, 1995, Proceedings, International Conference on (1995); Schultz, R. R. and Stevenson, R. L., “Extraction of High-Resolution Frames from Video Sequences,” IEEE Trans. Signal Processing 5(6), 1996; Borman, S. and Stevenson, R. L., “Super-resolution from image sequences—A review,” in Proc. 1998 Midwest Symp. Circuits and Systems, 1999, pp. 374-378; and Alam, M. S., Bognar, J. G., Hardie, R. C., and Yasuda, B. J., “Infrared Image Registration and High-Resolution Reconstruction Using Multiple Translationally Shifted Aliased Video Frames,” IEEE Trans. Instrumentation and Measurement, Vol. 49, No. 5 (2000)) and for cases where the focal plane for a still image is translated through a deterministic (or random) motion during the capture of a series of image captures (see, e.g.: Hochman, G., Yitzhaky, Y. Kopeika, N. S., Lauber, Y., Citroen, M., and Stern, A., “Restoration of Images Captured by a Staggered Time Delay and Integration Camera in the Presence of Mechanical Vibrations,” Applied Optics Vol. 43 No. 22, P. 4345-4354 (2004); and Haik, O. and Yitzhaky, Y., “Superesolution reconstruction of a video captured by a translational vibrated staggered TDI camera,” Proc. SPIE, 5558 (2004)).
NASA's Drizzle algorithm applies super-resolution techniques to reconstruct images taken with the wide-field cameras on the Hubble Space Telescope. See, e.g., Fruchter, S. A. and Hook, R. N., “Drizzle: A Method for the Linear Reconstruction of Undersampled Images,” PASP 114:144-152 (2002). See also: U.S. Pat. No. 5,341,174 to Xue et al., entitled “Motion Compensated Resolution Conversion System”; U.S. Pat. No. 5,696,848 to Patti et al., entitled “System for Creating a High Resolution Image from a Sequence of Lower Resolution Motion Images”; U.S. Pat. No. 5,920,657 to Bender et al., entitled “Method of Creating a High Resolution Still Image Using a Plurality of Images and Apparatus for Practice of the Method”; U.S. Pat. No. 5,946,426 to Carlebarch, entitled “Efficient Upward Raster Conversion Of Rotated Images”; U.S. Pat. No. 6,023,535 to Aoki, entitled “Methods and Systems for Reproducing a High-Resolution Image from Sample Data”; U.S. Pat. No. 6,208,765 B1 to Bergen, entitled “Method and Apparatus for Improving Image Resolution”; U.S. Pat. No. 6,535,650 B1 to Poulo et al., entitled “Creating High Resolution Images”; U.S. Pat. No. 7,085,323 B2 to Hong, entitled “Enhanced Resolution Video Construction Method and Apparatus”; and U.S. Pat. No. 7,352,919 B2 to Zhou et al., entitled “Method and System of Generating a High-Resolution Image from a Set of Low-Resolution Images”.
Super-resolution reconstruction has been used with line scan and TDI imagers where the focal plane consists of two imaging arrays with a sub-pixel offset between the pixel locations in one array and the locations in the other, as shown in FIG. 1. See, U.S. Pat. No. 7,227,984 B2 to Cavan, entitled “Method and Apparatus for Identifying the Defects in a Substrate Surface by using Dithering to Reconstruct Under-Sampled Images”; Grycewicz, T. J., Cota, S. A., Lomheim, T. S., and Kalman, L. S., “Focal plane resolution and overlapped array TDI imaging,” Proc. SPIE 708703 (2008). The overlapped array scheme has been implemented for the 2.5 m GSD “supermode” on the ESA SPOT-5 imaging satellite. See, Jacobsen, K., “High-Resolution Imaging Satellite Systems,” 3D-Remote Sensing Workshop, Porto (2005), accessed at http://www.ipi.uni-hannover.de/uploads/tx_tkpublikationen/HRIjac.pdf; and Poon, J., Smith, L., and Fraser, C., Orthoimage Resolution and Quality Standards, Project Number 2.3 Final Report, CRC for Spatial Information, University of Melbourne (2006).
Visible light imagers generally use a CCD array specifically designed for TDI imaging. In conventional TDI processing, the image motion across the array must be parallel to the array columns in order to avoid image smear. Likewise, the image motion across the array must be precisely locked to the array line rate to avoid smear. See also, Bodenstorfer, E., Fürtler, J., Brodersen, J., Mayer, K. J., Eckel, C., Gravogl, K., Nachtnebel, H., “High-speed Line-Scan Camera with Digital Time Delay Integration,” Proc. SPIE, 64960I (2007).
It would be useful to be able to improve the resolution achieved by a digital TDI camera. It would be useful to remove the constraint of precise synchronization of image motion and FPA timing required by conventional TDI, which would allow relaxation of the pointing and control requirements normally associated with a TDI camera. It would be useful to be able to reduce the blur caused by image drift in a TDI camera. It would be useful to be able to use a conventional video camera as a low-speed TDI imager, extending utility of the video camera for low-light imaging. It would also be useful to be able to use a conventional color video camera as a TDI imager to improve resolution and extend the image amplitude resolution and sensitivity at each pixel.